Clustering Words with the MDL Principle

Clustering Words with the MDL Principle

We address the probhml of automaticMly constructing a thesaurus by clustering words based on corpus data. We view this problem as that of estimating a joint distribution over the (:artesian product of a parti t ion of a set of nouns and a partition of a set of verbs, and propose a learning a.lgorithm based on the Mininmm Description Length (MDL) Principle for such estimation. We empirically com...

MDL Regularizer: A New Regularizer based on the MDL Principle

This paper proposes a new regularization method based on the MDL (Minimum Description Length) principle. An adequate precision weight vector is trained by approximately truncating the maximum likelihood weight vector. The main advantage of the proposed regularizer over existing ones is that it automatically determines a regularization factor without assuming any specific prior distribution with...

"clustering Words" Clustering Words Clustering Words

An Introduction to the MDL Principle

The MDL (Minimum Description Length) principle for statistical model selection and statistical inference is based on the simple idea that the best way to capture regular features in data is to construct a model in a certain class which permits the shortest description of the data and the model itself. Here, a model is a probability measure, and the class is a parametric collection of such model...

Constructing semantic representations using the MDL principle

Words receive a signiicant part of their meaning from use in communicative settings. The formal mechanisms of lexical acquisition, as they apply to rich situational settings, may also be studied in the limited case of corpora of written texts. This work constitutes an approach to deriving semantic representations for lexemes using techniques from statistical induction. In particular, a number o...